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ABSTRACT 

A study evaluated the Michigan Test Battery as a 
placement tool and indicator of student performance at the college 
level for one continuing education intensive academic English 
program. Three issues were addressed: whether the test: (1) is being 
used correctly for program placement and exit; ^2) is an accurate 
indicator of performance for placement purposes; and (3) shows 
progress in student scores over time. Test scores and initial 
placement for 53 svadents were compared with score interpretation 
guidelines to determine the frequency with which guidelines were 
followed in placement- decisions. In addition, placement test scores 
were compared with first quarter grades, and scores from four 
sequential test administrations were compared. Results of the 
analyses suggest that the program does not use the test correctly in 
many cases, the test is not an accurate indicator of performance in 
the level of placement, and the test does not measure student 
progress accurately. It is recommended that programs using the test 
for placement of advancement examine their objectives for test 
administration and their choice of test for the situation. The 
guidelines for test scoring are appended. (Contains 3 references.) 
(HSE) 
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Making dlfflcul*^; <? visions: Can the Michigan Test help? 

Abs\.iact 

The purpose of this study was to evaluate the use of the 
Michigan Test as a placement tool and as an Indicator of student 
performance for one Intensive academic English program. Three 
simple questions were posed: 1) Is the test used correctly?, 2) 
Is the Michigan Test an adequate placement tor.l? (is the test an 
accurate Indicator of performance In the level of placement?)^ 
and 3) Does the test show progress In student scores over time? 
The results of this analysis force us to question the use of the 
Michigan Test In making placement decisions. 
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Making difficult decisions: Can the Michigan Test help? 

To effectively manage lEPs, administrators must place students 
in class levels according to their proficiency in at least four 
language skill areas: speaking, reading, writing, and listening. 
Placing students accurately is a difficult task. This difficulty 
is due in part to the increasing number of placement tests 
available for this task and the lack of published empirical 
investigations of the validity and reliability of these tests. 

The Michigan Test (University of Michigan, 1968) is used fos 
placement and proficiency measurement In English as a Second 
Language (ESL) programs throughout the United States. The test 
consists of a listening component (Michigan Test of Aural 
Comprehension, or MTAC), a grammar and vocabulary component 
(Michigan Test of English Language Proficiency, or MTELP) and a 
thirty-minute writing sample (topics for which are developed by 
individual institutions). The test components are retired 
sections of the secured Michigan Battery. Except for the writing 
sample, the test consists of multiple choice questions. The 
ultimate objective of Michigan Test use Is to determine whether a 
non-native English speaker has adenuate proficiency to succeed in 
studies at the college or university level. 

Literature on the Michigan Test suggests that there are many 
problems associated with Its use, from lack of test security to 
lack of scientifically founded validity and reliability (Jenks, 
1987). Jenks (1987) warns that the test Is outdated and misused; 
however, he does note that a strength of the test Is the length 
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of time it has been in use, which has permitted institutions to 
compile years' worth of data. Such data will be used to test the 
hypotheses of this study. 

In the continuing education Program in ESL (PESL) from which 
these data were collected, the Michigan Test is used as an 
initial placement tool and as an exit test at the end of each 
quarter; however, scores at the end of the semester are not 
generally used to determine placement in successive levels. 
Purpose 

The purpose of this study was to evaluate the use of the 
Michigan Test as a placement tool and as an indicator of student 
performance for one intensive academic program (the PESL). Three 
simple questions were posed: 

1) Is the test used correctly? 

2) Is the Michigan Test an adequate placement tool? 

(Is the test an accurate indicator of performances in 
the level of placement?) 

3) Does the test show progress in student scores over time? 
Methods and Results 

Question One - Is the Michigan Test used correctly? 

The administration manual of the Michigan Test contains score 
interpretation guidelines for placement recommendations. Because 
these guidelines were constructed for student placement in 
regular university programs, they are not adequate for ESL 
programs using the test as placement for low and intermediate 
level students. Therefore, the PESL used these guidelines as a 
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baisis for the development of more extensive placement 
recommendations (see Appendix for complete score 
interpretation/placement guidelines). It is assumed that faculty 
and administrators Involved in the placement process follow 
these guidelines. This assumption will be tested below. 
Sample . Data for this analysis were stratified Into two levels 
out of six possible levels In the program. Placement test scores 
were compiled for twenty-four students Initially placed In Level 
Two (High Beginner) and twenty-nine students with Intlal 
placement In Level Four (High Intermediate). The Level Two 
population consisted of all students placed In Level Two 
commencing In the Fall quarter of 1988 who remained in the 
program for at least three quarters. The Level 4 population 
consisted of all students placed Into Level 4 In the Fall and 
Winter quarters of 1989 and the Winter of 1990. Although 
different versions of the test were used In different quarters, 
the score Interpretation guidelines remain constant across 
versions. 

Method . Michigan Test scores and Inlal placement were checked 
against the score interpretation guidelines to determine the 
frequency with which the guidelines were followed In placement 
decisions. In order to be placed correctly according to the 
guidelines, a Level Two student would have a cumulative test 
score fiom 38-47, and a Level Four student would have a 
cumulative test 7Core from 57-64 (on a one hundred point scale). 
Figure One shows the results of this comparison, with y 
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Indicating that th« guidelines were followed and n indicating 
that they were not. 



Level 2 (ns24) 



y= 21/24 



87.5% 



n= 3/2 J 



12.5% range of scores: 35-48 



Level 4 (na29) 



y=17/29 



56.6% 



n=12/29 



41.3% range of scores: 50-72 



Figure 1. 



In Level Two placement decisions, 12.5% o£ the decisions did 
not follow the guidelines, scores for these 12.5% of the 
students ranged from three points below the minimum score to one 
point above the maximum. In Level Foi:r, student placement fell 
outside of recommended guidelines 41.3% of the time. Students 
with scores from seven points below the minimum to eight points 
above the maximum were placed in Level Four. 

Discussion . The 112.5% of Level Two students placed out of the 
recommended level may be acceptable given the standard error of 
measurement for the test (3.54). However, the 41.3% of Level Four 
placement decisions which did not follow the guidelines are not 
as easUy explained. The range of difference is significant 
because the average range of points within one level Is 8. 
Therefore, according to the composite scores for these students, 
the English proficiency of the students placed in Level Four 
ranged from Level Three to Level Five. 

Within both the "correct" and "incorrect" decisions, according 
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to the average score designated here as "cumulative**) on the 
MTELP, MTAC, and writing sample, individual scores vary greatly. 
Figure Two shows examples o£ this variation. The number in 
parentheses is the recommended level for that score. 

Actual 







MTAC 


MTELP 


Writinq 


Cumulative 


Placement 


Student 


1 


72 (5) 


80 (6) 


53 (3) 


68 (5) 


Level 4 


student 


2 


66 (5) 


78 (6) 


57 (4) 


67 (5) 


Level 4 


Student 


3 


46 (2) 


75 (6) 


56 (3) 


59 (4) 


Level 4 


Student 


4 


41 (2) 


19 (1) 


50 (3) 


37 (1) 


Level 2 


Student 


5 


33 (1) 


56 (3) 


43 (2) 
Figure 2 


44 (2) 

• 


Level 2 



In the majority o£ cases, the placement decision was based 
most heavily on the writing sample score. For administrative 
purposes, it is clear that an equitable compromise between scores 
must be reached. However, at this point there is no evidence to 
suggest that weighting the writing sample score the most heavily 
is the most accurate solution. 

Question Two - ts the test an accurate indicator o£ performance? 

The assumption here is that there is a strong positive 
relationship between Michigan Test score and course performance. 
For example, it is expected that students with higher placement 
test scores are more successful in class, and those with lower 
scores are less successful. Performance is operationalized in 
this instance as grades. If, in fact, the Michigan Test is a 
useful placement tool for this program and is an accurate 
predictor of student performance, the correlation between test 
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scores and grades will be positive and high. 

Sample . The same stratified sample was used for this question as 
was used In Question One. To control for level, two separate 
correlational analyses were conducted. In addition to individual 
and cumulative test scores for each student, grades from the 
first quarter of study (ten weeks) were compiled. Although some 
students In both levels had been Incorrectly placed according to 
the score Interpretation guidelines, all students in the levels 
were working toward the same course goals. The Incorrect 
placement, however, made it more likely that a high correlation 
between test scores and grades would be seen because it resulted 
in a wider range of test scores for students In these levels. 
Method . Pearson correlations were calculated for the data In 
order to see whether a relationship existed between placement 
test scores and subsequent first quarter grades. 
Discussion . Figure Three shows the correlation between 
individual/cumulative plac^nent test scores and Individual/ 
cumulative grade point average for the first quarter of study. 





Level 


Two 








MTAC 


MTBLP 


WRITING 


CUM. 


Speaking/Listening 


0.145 


-0.264 


0.179 


-0.015 


Readlng/Vocab. 


-0.163 


-0.146 


0.366 


0.139 


Grammar /Writing 


-0.302 


0.039 


0.122 


-0.090 


Cumulative GPA 


-0.024 


-0.128 


0.265 


0.014 




Level 


Pour 








MTAC 


MTELP 


WRITING 


CUM. 
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Speak in^/Llstenlng 


0.187 


-0.064 


0.222 


0. 


160 


Reading/Vocab. 


-0.148 


0.328 


-0.056 


0. 


074 


Grammar /Writing 


0,063 


0.483 


0.073 


0. 


343 


Oimulatlve GPA 


0.033 


0.272 


0.096 


0. 


219 



Figure 3. 



Correlations between the placement test scores and the grades 
were low, suggesting little or no relationship. However, In Level 
Four, where over 40% of the students were placed by some method 
other than following the score Interpretation guidelines, the 
correlations for cumulative test scores and grades were somewhat 
higher. The highest correlation between cumulative test score and 
grades was that for the grade in the grammar /writing class. 
Question 3 — Does the test show progress In student scores over 

time? 

It is not possible to test whether the Michigan Test measures 
student progress accurately. We have no accurate measure of 
progress with which to compare it. However, it is useful to 
examine whether the test indicate? any student progress; that 
is, whether student scores progress. For the sake of analysis. 
It was assumed that 

1) The test Is an adequate Indicator of pertormance for students 
as a whole. 

2) students who take the test ar«5 motivated to try their best. 

3) Performance Improves in the program. 

If these assumptions are true, then It Is expected that student 
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scores would Increase consistently over time. If this Is found to 
be the case, then the Michigan Test may be said to at least 
measure student progress In some form. 

Sample . The sample for this question consisted of forty-two 
foreign students enrolled in the PBSL between the Fall 1988 
quarter and the Fall 1989 quarter. These forty-two students 
compose the entire population of students completing three 
quarters of study In the program during this period for whom 
complete data is available. Five students were excluded because 
of missing data. Figure Four shows the breakdown of students by 
native country. 



Japan 


28 


United Arab 


Bmlirates 3 


Lebanon 


1 


Taiwan 


2 


Thailand 


1 


"orea 


4 


Hong Kong 


1 


Switzerland 


1 


Venezuela 


1 




Figure 



Each of these students took the Michigan Test at least four 
times - once for Intlal placement and once at the end of each of 
three successive quarters. Some of the students repeated an MTAC 
form Q'\ce (after the Mzd successive quarter). None of the 
students repeated an MTELP or writing sample. 
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Method . Scores for the four administrations of the Michigan Test 
were compiled. Mlnlmuras, maximums, and means are provided In 
Tables 1-4. "Progress" by quarter was determined for each 
student by subtracting test scores from each quarter from test 
scores from the subsequent administration of the test. Cumulative 
progress was determined by su^tractlng the placement test scores 
from the scores for the fourth administration of the test. 

Discussion . An anallysls of mean overall progress suggests that 
students Improved an average of 17.85 points over the four test 
administrations (equal to three quarters, or 30 weeks of ESL 
Instruction). However, further analysis suggests that thes<i 
mean values are misleading. Table 1 Indicates that 27 out of the 
42 students (64.2%) were at or below this mean. Furthermore, 
student progress varied widely In each quarter. In addition, 
several Incidents of negative gains are documented In the T^^bles. 
Conclusions 

Question 1 - Is the test used correctly? 

Out of the Level 2 students In the sample, 12.5% were not 
placed within the level recommended by the guidelines. Within 
the Level Four sample, this number rose to 41.3%. Student scores 
on Individual tests ranged widely; however, the writing sample 
score was most likely to affect student placement. From these 
reaults, ve conclude that. In many cases, the Michigan Test was 
not used correctly - that Is, according to the guidelines for 
score Interpretation. 
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Question 2 - Is the test an accurate Indlca^cr o£ performance? 

Several factors may account for the lack of correlation 
between test scores and grades. Jenks (1937) suggests that, 
because the MTELP Is based on structuralist principles, it may 
not measure proficiencies taught by other methods. Jones (1987) 
claims that the NTAC tests oral grammar rather than auditory 
discrimination, the latter of which Is stressed In the PESL 
curriculum; therefore, student grades may be based on entirely 
different performance objectives than the test. The results 
Indicate that there Is little or no relationship between placment 
test scores and first quarter grades. We therefore conclude that 
the test is not an accurate predictor of student performance as 
measured by grades. 

Question 3 - Does the test show progress In student scores over 
time? 

According to the score Interpretation guidelines used In the 
PBSL, an average of 7.6 points is necessary to move from one 
program level to the next (e.g., from Low Intermediate to High 
Intermediate). Therefore, from the cumulative placement test 
score to the final test score after three quarters, students 
should gain approximately 22.8 points. The students In this 
sample progressed through the program levels, but most did not 
achieve an average gain of 22.8 points. In addition to the lack 
of appropriate progress (as measured by test scores), wide 
variations and large negative gains In the progress of 
individual students are evident. These data Indicate that the 
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assumptions made concerning student motivation, performance 
Improvement within the program, and test adequacy are faulty; 
however, it must be noted that students do not progress within 
the program unless they improve. This implies that either 1) the 
students are not motivated when they take the test, or 2) the 
test is not an adequate indicator of student performance. In 
either case, it can be Inferred that the test does not measure 
student progress accurately. 

Implications 

The findings of this study suggest the following: 

1) The PESL does not use the Michigan Test correctly in many 
cases . 

The score interpretation guidelines are based on a straight 
average of the three component test scores. However, PBSL 
faculty and administrators sometimes weighted the writing sample 
score more heavily that the scores from the other two components. 
Perhaps it would be useful to calculate the cumulative score in a 
way other than a straight average. For example, the PBSL could 
conduct multiple regression analyses to determine a more 
appropriate weighting of the component scores. 

2) The Michigan Test in not an accurate indicator of 
performance in the level of placement. 

One possible explanation for this finding, as Jenks (1987) 
proposed, is that both the content and construct validity of the 
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test are questionable. Changes In language teaching theory and 
practice are not reflected In the test. Therefore, students are 
being evaluated on knowledge which Is noc part of the PESL 
curriculum. Alternatively, the test may attempt to measure the 
correct knowledge base, but It does so Inappropriately In light 
of advances In evaluation techniques since 1968. 

3) The Michigan Test does not measure student progress 

accurately. 

Problems In test administration may account for a lack of 
motivation on the part of the students wh 'ke the test. Because 
the PESL does not use the exit test scores, students may become 
apathetic about their performance on the exit tests. Therefore, 
the Program should rethink Its use of the exit tests. • 

If, however, the problem lies with the adequacy of the test as 
an Indicator of student progress, then the content and construct 
validity issues raised above are relevant for progress as well as 
performance 

This study suggests that the Program in ESL and other programs 
using the Michigan Test for placement and/or advancement purposes 
must examine their objectives for test administration. In 
addition, they must evaluate whether the Michigan Test is the 
most appropriate tool to use in meeting these objectives. 
Of course, this study is not without its limitations; it was 
focussed on a relatively small sample within one ESL program. 
Further research is necessary to assess the value of the Michigan 
Test to ESL programs. 
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GUIDELINES FOR SCORING OF MICHIGAN TEST I S S AY 



University Level; Native command of Enttllsh 
Excellent organization and expression 

University Level; Very Rood cotanand of English 
Style; Interesting to read 

Good dlscusiiion of topic rather than mere description 
Organization; Logical progression of Ideas and paragraphs 

Good topic sentences 
Mechanics; Excellent punctuation, capitalization and spelling 
Grammar; Excellent use of transition words 

Well-controlled variation of sentence structure 
Use of complex clauses 
No article or preposition errors 
Vocabulary: Almost no errors In parts of speech 

Varied and appropriate use of content and expressive 
vocabulary 

Level 6 or Community College with Special E nglish : Good Command 

of Engllsli 

Style: Interesting to /ead 

Some discussion in addition to basic description of topic 
Organization: Ideas organized In paragraph font with good topic 

sentences, supporting sentences and conclusion 
Mechanics: Good punctuation, capitalization, spelling 
C axnaai : Good use of transitional expressions 

Ability to use all cense forms In appropriate context 
Use of modals, gerunds and Infinitive constructions 
Use of clauses, conditionals and comparisons 
Infrequent syntax errors 
Vocabulary: Few errors In parts of speech 
Use of expressive vocabulary 

Level 5 : Above Average Connand of English 
Style Oranglzatlon, Mechanics: Emphasizes description with 
discussion 

Identifiable progression of Ideas 
Use of paragraph form plus Indantatlon 
Above average punctuation, capitalization, spelling 
Grasanar: Good command of tenses 

Appropriate use of simple and continuous present, simple 

past, present perfect, future, conditionals, passive voice 
Very few run-on sentences and fragments 

Use of conjunctions (and, or, but) and use of subordlnators 

In complex sentences 
Very few article errors 
Vocabulary: Good command of basic vocabulary 
A few errors in parts of speech 
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Levi 4 ; Avcra n* Coimi nd of EntlUh 

^'iil* M.ch«nlct: U.t of paragraph fonn. Including 

Indentation, topic sentences, related sentences and conclusion 
Correct capitalization and punctuation conclusion 
Average spelling 
Cratnoar: Agreement of all tenses in paragraph 
Subject-verb *greement 
Limited use of compound sentences 
Some article errors 
Some run-on sentences and fragments 
Vocabulary: Average cotmand of basic vocabulan.- 
Some errors in parts of speech 

^^'^^^ 3 ; Below Avera ge Co mmand of En g lish 

Style. Organliatlon. Mechanics: Follows cc.p.^iMon directions 
Uses paragraph torm, Including indent^cloo 
Some evidence of topic and supporting sentences 
Legible handwriting 
Basic capitalization, punctuation 
Frequent spelling mistakes 

Crainmar: Limited use of simple present, past and future with be + 
golnf^ to — 

Some use of adverblals of time aad place 
borne errors in word order 
Some mistakes In tense and agrecaeot 
Use of mainly simple sentences 
Vocabulary: Below average command of basic vocabulary 

Some understanding of parts of speech but frequent 

errors In choosing correct foros 

Level 2; Negligible Comcand of Enal/ sh 

^'Ji^ Oj8*"i"tlon Mechanics: Almost no knowledge of composition 

ll^LilV ^"'^,<»^P*"8"Phlng, indentation, topic sentences 

Handwriting needs improvement 

Little use of capitalization 

Many punctuation and spelling mistakes 

Composition directions not followed 

Frequent incomplete sentences 
CraaBMr: Use of simple present only 
Vocabulary: Extremely lljnited 

Level l! Ho Command of Eni^llsh 

Ucks handritlng skills, but has knowledge oTZn.llsh alphabet 
and numbers ' 

A beginner in terms of gramnar, mechanics and vocabulary 
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